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PREFACE 


The  work  described  in  this  publication  was  performed  by  the 
Mathematical  Analysis  Research  Corporation  (MARC)  under  contract  to 
the  Jet  Propulsion  Laboratory,  an  operating  division  of  the  California 
Institute  of  Technology.  This  activity  is  sponsored  by  the  Jet 
Propulsion  Laboratory  under  contract  NAS7-918,  RE182,  A187  with  the 
National  Aeronautics  and  Space  Administration,  for  the  United  States 
Army  Intelligence  Center  and  School. 


This  specific  work  was  performed  in  accordance  with  the  FY-87 
statement  of  work  (SOW  #2). 
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The  Impact  of  Using  Truncated  Data  To  Estimate  Variance 


The  impact  of  using  truncated  data  to  estimate  variance  is  that  one's 
estimates  are  biased  small.  This  results  in  error  ellipses  that  are  too 
small.  The  amount  of  this  bias  depends  on  how  close  the  true  standard 
deviation  (-square  root  of  the  variance)  is  to  the  truncation  point.  It  can 
be  insignificant  or  very  significant.  What  about  real  systems? 

Guardrail  V:  An  angular  truncation  value  of  ^.5  degrees  is  used 
in  determining  which  data  is  used.  The  fix  method 
used  is  a  3~dimenjional  weighted  perpendicular. 

An  estimated  covariance  is  used  with  an  F  cutoff. 

Review  of  this  method  did  not  raise  any  major  concerns 
because  MARC  was  led  to  believe  that  the  true  standard 
deviation  of  incoming  data  would  be  much  smaller  than 
*».5  degrees. 

Quickfix:  An  angular  truncation  value  of  ^  degrees  (possibly  6 

was  intended  as  there  is  a  contradictory  comment  in  the 
computer  code  used  in  analysis  of  the  algorithm) 
determines  which  data  is  used.  The  fix  method  used 
Is  a  2-dimensional  weighted  perpendicular.  A  standard 
deviation  of  3  degrees  is  used  unless  the  estimated 
standard  deviation  Is  bigger.  If  it  Is  bigger  then 
the  estimated  standard  deviation  is  used.  Thus  the 
estimated  standard  deviation  is  only  used  when  it  is 
close  to  the  truncation  point. 

Analysis  of  Quickfix  suggests  a  need  to  a  understand  how  truncation  affects 
estimates  of  covariance  (and  hence  standard  deviation).  A  full  analysis  would 
require  a  bigger  study  than  will  be  performed  here,  nonetheless  the  basic 
impact  of  truncation  is  can  be  understood  in  terms  of  the  following 
observations: 

A)  The  existence  of  code  for  the  large  sigma  case  in  Quickfix  suggests 
the  possibility  that  either 

1)  large  sigma  are  possible  and  the  truncation  value 
can  be  close  to  the  cutoff  value 
or  2)  the  assumption  of  normality  is  suspected  and  the  code 

was  designed  to  cover  the  "fat  tails"  problem.  The  "fat 
tails"  problem  occurs  when  the  probability  of  a  large  error 
Is  higher  than  predicted  by  the  normal. 

B)  One  would  seldom  invoke  the  special  code  in  Quickfix  except  in  the 
small  sample  size  case.  (The  small  sample  size  case  invokes  it  more 
frequently  both  because  the  estimator  for  variance  is  computed  dividing 


by  n~3  instead  of  n-2  and  because  variance  of  the  estimate  of  variance 
calculation  is  greater  in  this  case.)  Representative  cases  are 
computed  in  Tables  I  and  II  below. 

C)  Truncated  normals  produce  low  estimates  of  variance.  See  table  III. 
The  division  by  n-3  in  the  Quickfix  variance  calculation  instead  of  n-2 
has  a  corrective  impact  in  the  small  sample  size  case.  See  Tables  IV 
and  V. 

CALCULATIONS 

The  calculations  are  meant  to  be  suggestive.  Simplifying  assumptions 
have  been  made  to  allow  this  study  to  be  performed  without  interfering  with 
other  work.  In  Tables  I, 111,1V  and  V  the  primary  simplifying  assumption  is 
that  the  truncation  is  centered  about  the  true  mean.  This  assumption  is 
really  only  acceptable  for  the  large  sample  size  case.  A  formula  for 
recomputing  tables  such  as  Tables  III,  IV,  and  V  may  be  found  in  Appendix  I. 
The  reason  for  not  including  these  here  is  that  MARC  does  not  have  enough 
background  on  the  cases  of  interest  to  provide  narrow  down  the  cases  computed 
to  a  small  enough  set  for  inclusion.  Similarly  Table  I  may  be  recomputed  for 
with  truncation  not  centered  about  the  mean  by  adding  a  constant  to  X  in  the 
computer  program  attached  in  Appendix  II. 

TABLE  I 

Percentage  of  Quickfix  Estimates  That  Are  Greater 
Than  Three  Degrees  Assuming  An  Underlying  Normal 

Number  of  Lines  of  Bearing  True  Standard  Percentage  of 

Used  In  the  Estimate  Deviation  Estimates  Greater 
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TABLE  II 

Percentage  of  Quickfix  Estimates  That  Are  Greater 
Than  Three  Degrees  Assuming  Uniform  Error 


Number  of  Lines  of  Bearing 
Used  In  the  Estimate 


Percentage  of 
Estimates  Greater 


Note  that  uniform  may  be  considered  to  be  normal  with  very  high  sigma  or  wide 
tailed  normal  exaggerated.  Furthermore  centering  at  the  true  mean  does  not 
matter.  Uniform  is  probably  a  bound  on  the  optimistic  side. 

Tables  III,  IV  and  V  which  follow  overestimate  the  impact  somewhat  as 
they  assume  the  truncation  is  centered  about  the  true  mean.  Since  the  true 
mean  is  not  known  this  assumption  is  not  warranted.  An  asymmetric 
distribution  expression  is  derived  in  the  appendix  that  can  be  used  to 
determine  sensitivity  to  lack  of  knowledge  of  the  true  mean. 

TABLE  III 

Reduction  In  Expected  Variance  Because  of  Truncation 

GENERAL  CASE 


Truncation  Point 
In  Standard  Deviations 


Expected  Value  of  the  Computed  Variance 
As  A  Percentage  Of  The  True  Variance 

.9939 
.9733 
-91 13 
.7738 
.5515 
.2910 
.0808 


TABLE  IV 

Reduction  In  EXPECTED  Variance  Because  of  Truncation 
SPECIAL  CASE-QUICKFIX  Truncation  At  U  Degrees 


True  Standard  Deviation 
of  Angular  Error _ 

10  Degrees 
5  Degrees 
H  Degrees 


Expected  Value  of  the  Computed  Variance 
As  A  Percentage  Of  The  True  Variance 

.0521 

.1956 

.2910 


QUICKFIX  does  not  use  the  standard  variance  formula  however.  Where  one 
normally  divides  by  n-2  where  n  is  sample  size,  QUICKFIX  appears  to  divide  by 
n~3.  It  is  unclear  if  the  use  of  n~3  was  an  attempt  to  compensate  for 
truncation.  in  any  case  using  this  adjustment  one  can  compute  values  such  as 
those  in  Table  V  that  show  the  net  effect  of  truncation  and  dividing  by  the 
wrong  normalization  constant,  l.e.  n-3  instead  of  n-2. 
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TABLE  V 

Reduction  In  COMPUTED  Variance  Because  of  Truncation 
SPECIAL  CASE-QUICKFIX  Truncation  At  H  Degrees 

True  Standard  Deviation  Expected  Value  of  the  'QUICKFIX'  Variance 

of  Angular  Error  As  A  Percentage  Of  The  True  Variance 

_ _ n-fl _ n-5 _ . . . _ n-» _ 

10  Degrees  .11*11  .0781  .0521 

5  Degrees  .3912  .293*1  .1956 

Degrees  .5821  .H3&5  .2910 

For  small  sample  size  the  fact  that  the  variance  is  not  being  computed  about 
the  true  mean  would  also  raise  these  values  somewhat.  For  small  sample  sizes 
and  standard  deviations  close  to  3  degrees,  the  fact  that  3  degrees  is  the 
minimum  allowed  would  increase  the  percentage  as  well.  A  complete  analysis  of 
the  ways  these  factors  interact  would  require  simulation.  There  does  not 
appear  to  be  any  reason  to  increase  the  sample  size-H  case  more  than  for  other 
sample  sizes,  however.  Thus  it  is  our  opinion  that  adjusting  all  sample  sizes 
the  same  amount  would  be  preferable  to  QUICKFIX’s  approach  of  dividing  by  n~3. 


appendix  I 


Truncated  Estimates  of  Variance 
Without  loss  of  generality  assume  X-N(0,o2). 

For  ease  of  notation  assume  N(x)  denotes  the  value  of  the  cumulative  density 
of  the  N(0, 1 )  normal  at  x. 

Then  ViRTruncated(x)  '  ETruncated<x!> 

b  2  2  2 

fa  x  exp(-x  /  (2a  ))  dx 

/k  exp(-x2/(2o2) )  dx 

-  o2{1  -  [1  A/2tt3[ (b/o)exp(-(b/o)2/2) )-(a/o)exp(-(a/c)2/2) ) ]/[N(b/o)-N(a/o) ]} 

The  table  shown  in  this  memo  is  for  a=-b. 


